Sally the Congressperson: A Psycho- and Sociolinguistic Investigation into the Relationship Between Ideology, Gender, and Language: Final Production Model

1 Introduction

The code and information contained herein constitutes the complete write-up of the experiments I carried out for my first Qualifying Paper towards the PhD in Linguistics at Stanford University. The goal is to make this document both a dumping ground for my ideas while it is in progress, as well as, eventually, a publicly-available version of my Qualifying Paper, in the spirit of Open Science.

1.1 Preliminaries

1.1.1 Setting up the Notebook

For this write-up and analysis, I require the following packages, loaded in here:

I also use a custom color palette, so I include the code for that here as well¹.

bran_palette = c("#7ae7e5", "#fe5f55", "#B2A6DE", "#14342b", "#69385c")
sec_palette = c("#3d405b","#e07a5f","#81b29a","f2cc8f")


theme_set(theme_minimal())

1.1.2 Frequency Data

We also need the frequency data! These frequency values are taken from the Corpus of Contemporary American English² (COCA), from only the spoken part of the corpus.

frequency <- read.csv("freq_vals.csv")

Here we set up objects which will specify both the liberal news sources (lib_cols) and the non-independent news sources (non_inds). These news sources’ left vs. right wing skews were coded according to their positions on the Ad Fontes Media Bias Chart³.

lib_cols <- c('ABC','CNN','PBS','NBC','MSNBC','NPR','CBS')
non_inds <- c('ABC','CNN','PBS','NBC','MSNBC','NPR','CBS','FOX')

Here we take the raw frequency values obtained from COCA and turn them into words per million values, to account for the varying sizes of the media sources in the corpus. We also code the terms for whether or not the are gender neutral, and whether the neutral forms are compounds (e.g. congressperson) or adoptions of the male form (e.g. actor). Finally, we code them for their form gender.

frequency <- frequency %>%
  mutate(total_left = rowSums(frequency[lib_cols])) %>%
  mutate(total_right = FOX) %>%
  mutate(all_wpm = ((total_left + total_right) / 121500000) * 1000000) %>%
  mutate(left_wpm = (total_left/109300000) * 1000000) %>%
  mutate(right_wpm = (total_right/12200000) * 1000000) %>%
  mutate(neutral_binary = ifelse(gender=="neutral",1,0)) %>%
  mutate(morph_type = ifelse(lexeme!= 'actor' & lexeme!= 'host' & lexeme !='hunter' & lexeme!= 'villain' & lexeme!= 'heir' & lexeme!= 'hero','compound','adoption')) %>% 
  rename(form = word) %>% 
  mutate(freq_gender = ifelse(form == "actress" | form == "anchorwoman" | form == "stewardess" | form == "businesswoman" | form == 'camerawoman' | form == 'congresswoman' | form == 'craftswoman' | form == 'crewwoman' | form == 'firewoman' | form == 'forewoman'  | form == 'heiress' | form == 'heroine' | form == 'hostess' | form == 'huntress' | form == 'laywoman' | form == 'policewoman' | form == 'saleswoman' | form == 'stuntwoman' | form == 'villainess' | form == 'weatherwoman',"female",ifelse(form == "anchor" | form == "flight attendant" | form == "businessperson" | form == 'camera operator' | form == 'congressperson' | form == 'craftsperson' | form == 'crewmember' | form == 'firefighter' | form == 'foreperson' | form == 'layperson' | form == 'police officer' | form == 'salesperson' | form == 'stunt double' | form == 'meteorologist',"neutral",ifelse(form == "anchorman" | form == "steward" | form == "businessman" | form == 'cameraman' | form == 'congressman' | form == 'craftsman' | form == 'crewman' | form == 'fireman' | form == 'foreman' | form == 'layman' | form == 'policeman' | form == 'salesman' | form == 'stuntman' | form == 'weatherman',"male",'neutral'))))

Because we will be working in log space, we need to re-write frequency to avoid zero numbers. However, we still want them to be small to represent their non-presence in the corpus, so we give them a very small WPM value.

frequency[frequency == 0.00000000] <- 0.0001

We then take the log WPM of each media type, and subset out the individual media sources, which are no longer informative to the questions we are asking.

frequency <- frequency %>%
  mutate(log_right = log(right_wpm), log_left = log(left_wpm), log_all = log(all_wpm))

frequency <- subset(frequency, select = -c(ABC,CNN,PBS,NBC,MSNBC,NPR,CBS,FOX))

Now we can create frequency values for the whole corpus, as well as the left vs. right wing media sources, named so that we can use them in later analyses. We do this by creating three gendered data frames, then joining them into net_lex, below.

freq_all <- frequency %>% 
  select(lexeme,freq_gender,all_wpm) %>% 
  pivot_wider(names_from = freq_gender,values_from = all_wpm,names_prefix="all_wpm_")

freq_right <- frequency %>% 
  select(lexeme,freq_gender,right_wpm) %>% 
  pivot_wider(names_from = freq_gender,values_from = right_wpm,names_prefix="right_wpm_")

freq_left <- frequency %>% 
  select(lexeme,freq_gender,left_wpm) %>% 
  pivot_wider(names_from = freq_gender,values_from = left_wpm,names_prefix="left_wpm_")

net_lex <- left_join(freq_all,freq_right,by="lexeme")

net_lex <- left_join(net_lex,freq_left,by="lexeme")

net_lex <- net_lex %>%
  mutate(morph_type = ifelse(lexeme!= 'actor' & lexeme!= 'host' & lexeme !='hunter' & lexeme!= 'villain' & lexeme!= 'heir' & lexeme!= 'hero','compound','adoption')) %>% 
  mutate(
    fem_odds = all_wpm_neutral / all_wpm_female,
    male_odds = case_when(
      morph_type == "adoption" ~ log(all_wpm_neutral / all_wpm_neutral),
      morph_type == "compound" ~ log(all_wpm_neutral / all_wpm_male) 
    )
  )

1.1.3 Norming Data

And here is the norming data values:

norming_data <- read.csv("norming_data.csv") %>%
  filter(id!="example1") %>% # Will filter out non-critical trials, i.e. the example trial from the beginning of the experiment
  mutate(equalized_response = ifelse(scale=="FM",8-response,response)) %>% # This will render all data points on the same scale, as participants randomly received either "very likely a man" or "very likely a woman" as the left end of their response scale, with the other appearing at the right end
  mutate(orthog = ifelse(orthog=="sroceress","sorceress",orthog)) %>% # Fixes a typo
  mutate(id = ifelse(id=="Stunt_double","stunt double",id)) %>% # This, as well as all lines below it, convert compounds formed by spaces from their underscore forms to their spaced forms (e.g. police_officer -> Police officer)
  mutate(id = ifelse(id=="Police_officer","police officer",id)) %>%
  mutate(id = ifelse(id=="Flight_attendant","flight attendant",id)) %>%
  mutate(id = ifelse(id=="Anchor","anchor",id)) %>%
  mutate(id = ifelse(id=="Businessperson","businessperson",id)) %>%
   mutate(id = ifelse(id=="Camera","camera operator",id)) %>%
  mutate(id = ifelse(id=="Congressperson","congressperson",id)) %>%
  mutate(id = ifelse(id=="Craftsperson","craftsperson",id)) %>%
  mutate(id = ifelse(id=="Crewmember","crewmember",id)) %>%
  mutate(id = ifelse(id=="Firefighter","firefighter",id)) %>%
  mutate(id = ifelse(id=="Foreperson","foreperson",id)) %>%
  mutate(id = ifelse(id=="Layperson","layperson",id)) %>%
  mutate(id = ifelse(id=="Meteorologist","meteorologist",id)) %>%
  mutate(id = ifelse(id=="Salesperson","salesperson",id)) %>%
  mutate(id = ifelse(id=="Actor","actor",id)) %>%
  mutate(id = ifelse(id=="Heir","heir",id)) %>%
  mutate(id = ifelse(id=="Hero","hero",id)) %>%
  mutate(id = ifelse(id=="Host","host",id)) %>%
  mutate(id = ifelse(id=="Hunter","hunter",id)) %>%
  mutate(id = ifelse(id=="Villain","villain",id)) %>%
  mutate(orthog = ifelse(orthog=="airline steward","steward",orthog)) %>%
  mutate(orthog = ifelse(orthog=="airline stewardess","stewardess",orthog))

norming_exclusion <- norming_data %>% 
  filter(gender=="female") %>% 
  group_by(workerid) %>%
  summarize(female_mean = mean(equalized_response)) %>%
  unique() %>% 
  mutate(exclusion = female_mean < mean(female_mean) - 2*sd(female_mean)) %>%
  filter(exclusion==TRUE)

norming_data <- norming_data[!(norming_data$workerid %in% norming_exclusion$workerid),]

norming_means <- norming_data %>%
  filter(neutral_morh !="male_adoption") %>%
  group_by(orthog,id) %>%
  summarise(indi_mean = mean(equalized_response), trial_count=n()) %>%
  rename(form = orthog) %>%
  rename(lexeme =id)

`summarise()` has grouped output by 'orthog'. You can override using the `.groups` argument.

norming_adoptions <- norming_data %>%
  filter(neutral_morh == "male_adoption") %>%
  group_by(orthog) %>%
  summarise(indi_mean = mean(equalized_response), trial_count=n()) %>%
  mutate(lexeme = ifelse(orthog=="actress","actor",ifelse(orthog=="heiress","heir",ifelse(orthog=="heroine","hero",ifelse(orthog=="hostess","host",ifelse(orthog=="huntress","hunter",ifelse(orthog=="villainess","villain",orthog))))))) %>%
    rename(form = orthog)

norming_adoptions <- norming_adoptions[, c("lexeme", "form", "indi_mean", "trial_count")]

norming_means <- rbind(norming_means,norming_adoptions) %>%
  rename(lexeme_norm = lexeme)

Neutrals Only

norming_means_neutral <- norming_data %>%
  filter(gender=="neutral") %>%
  filter(neutral_morh !="male_adoption") %>%
  group_by(orthog,id) %>%
  summarise(indi_mean = mean(equalized_response), trial_count=n()) %>%
  rename(form = orthog) %>%
  rename(lexeme =id)

`summarise()` has grouped output by 'orthog'. You can override using the `.groups` argument.

norming_adoptions_neutral <- norming_data %>%
  filter(gender=="neutral") %>%
  filter(neutral_morh == "male_adoption") %>%
  group_by(orthog) %>%
  summarise(indi_mean = mean(equalized_response), trial_count=n()) %>%
  mutate(lexeme = ifelse(orthog=="actress","actor",ifelse(orthog=="heiress","heir",ifelse(orthog=="heroine","hero",ifelse(orthog=="hostess","host",ifelse(orthog=="huntress","hunter",ifelse(orthog=="villainess","villain",orthog))))))) %>%
    rename(form = orthog)

norming_adoptions_neutral <- norming_adoptions_neutral[, c("lexeme", "form", "indi_mean", "trial_count")]

norming_means_neutral <- rbind(norming_means_neutral,norming_adoptions_neutral)

2 Analysis

2.1 Data Read-in

prod_data <- read.csv("production_data.csv")

2.2 Exclusions

prod_exclusion <- prod_data %>% filter(name=='attention') %>%
  group_by(workerid) %>%
  summarise(accuracy = mean(correct)) %>%
  mutate(exclude = ifelse(accuracy < 0.80,'Yes','No')) %>%
  filter(exclude == "Yes") 

prod_data <- prod_data[!(prod_data$workerid %in% prod_exclusion$workerid),]

2.3 Additional Information

gender_transcendence_cols <- c('subject_information.gender_q1','subject_information.gender_q2','subject_information.gender_q3','subject_information.gender_q4','subject_information.gender_q5')

gender_linked_cols <- c('subject_information.gender_q6','subject_information.gender_q7','subject_information.gender_q8','subject_information.gender_q9','subject_information.gender_q10','subject_information.gender_q11','subject_information.gender_q12','subject_information.gender_q13')

prod_data <- prod_data %>%
  mutate(gender_trans = 100 - (rowMeans(prod_data[gender_transcendence_cols]))) %>%
  mutate(gender_link = rowMeans(prod_data[gender_linked_cols])) 

gender_all = c('gender_trans','gender_link')

prod_data <- prod_data %>%
  mutate(gender_total = rowMeans(prod_data[gender_all]))

prod_data <- prod_data %>%
  filter(type == "critical") %>%
  mutate(response_gender = ifelse(response == "actress" | response == "anchorwoman" | response == "stewardess" | response == "businesswoman" | response == 'camerawoman' | response == 'congresswoman' | response == 'craftswoman' | response == 'crewwoman' | response == 'firewoman' | response == 'forewoman'  | response == 'heiress' | response == 'heroine' | response == 'hostess' | response == 'huntress' | response == 'laywoman' | response == 'policewoman' | response == 'saleswoman' | response == 'stuntwoman' | response == 'villainess' | response == 'weatherwoman',"female",ifelse(response == "anchor" | response == "flight attendant" | response == "businessperson" | response == 'camera operator' | response == 'congressperson' | response == 'craftsperson' | response == 'crewmember' | response == 'firefighter' | response == 'foreperson' | response == 'layperson' | response == 'police officer' | response == 'salesperson' | response == 'stunt double' | response == 'meteorologist',"neutral",ifelse(response == "anchorman" | response == "steward" | response == "businessman" | response == 'cameraman' | response == 'congressman' | response == 'craftsman' | response == 'crewman' | response == 'fireman' | response == 'foreman' | response == 'layman' | response == 'policeman' | response == 'salesman' | response == 'stuntman' | response == 'weatherman',"male",'neutral')))) %>%
  mutate(congruency = ifelse(gender == response_gender,"true","false")) %>%
  mutate(incongruent = ifelse(gender == "male" & response_gender == "female","incongruent_mtf",ifelse(gender == "female" & response_gender == "male","incongruent_ftm","real"))) %>%
  mutate(neutrality = ifelse(response_gender == "neutral","true","false"))%>%
  mutate(morph_type = ifelse(lexeme!= 'actor' & lexeme!= 'host' & lexeme !='hunter' & lexeme!= 'villain' & lexeme!= 'heir' & lexeme!= 'hero','compound','adoption')) %>%
  mutate(poli_party = ifelse(subject_information.party_alignment == 1 | subject_information.party_alignment == 2,'Republican',ifelse(subject_information.party_alignment == 4 | subject_information.party_alignment == 5,'Democrat','Non-Partisan'))) %>%
  mutate(response_neutral = ifelse(response_gender == "neutral",1,0)) %>% 
  mutate(young_old = ifelse(subject_information.age > 40,"old","young")) %>%
  rename(form = response) %>% 
  filter(!is.na(subject_information.age)) %>% 
  filter(!is.na(poli_party))

Joining independent data Now that we have this information, we want to left join our frequency information so that we can use it in later analyses. We begin by adding the norming values’ means, of all forms.

final_prod <- left_join(prod_data,norming_means,by="form")

Now we can add the frequency data as well, by form.

final_prod <- left_join(final_prod,freq_lex,by="lexeme")

Now we can add additional lexeme-level norming information for the neutral forms, so that we can tackle the question of what role real-world expectations have on processing times.

final_prod <- left_join(final_prod,norming_means_neutral,by="lexeme")

final_prod <- subset(final_prod, select = -c(form.y,trial_count.y,trial_count.x,error,subject_information.comments,subject_information.asses,subject_information.enjoyment,subject_information.gender_q1,subject_information.gender_q2,subject_information.gender_q3,subject_information.gender_q4,subject_information.gender_q5,subject_information.gender_q6,subject_information.gender_q7,subject_information.gender_q8,subject_information.gender_q9,subject_information.gender_q10,subject_information.gender_q11,subject_information.gender_q12,subject_information.gender_q13,lexeme_norm,subject_information.comments)) %>%
  rename(form_norm = indi_mean.x, lexeme_norm = indi_mean.y, form = form.x)

net_lex <- net_lex %>% 
  select(lexeme,fem_odds,male_odds)

final_prod <- left_join(final_prod,net_lex,by="lexeme")

2.3.1 Heavy data manipulation; DANGER ZONE

final_prod <- final_prod %>%
  mutate(log_odds = case_when(
    gender == "male" ~ male_odds,
    gender == "female" ~ fem_odds
  )) %>% 
  mutate(c_gender_total = scale(gender_total,scale=FALSE)) %>%
  mutate(c_gender_trans = scale(gender_trans,scale=FALSE)) %>%
  mutate(c_gender_link = scale(gender_link,scale=FALSE))

2.3.2 Before we subset out the incongruents

prod_all <- final_prod %>%
  mutate(poli_party = factor(poli_party,ordered = FALSE)) %>%
  mutate(trial_gender = factor(gender,ordered = FALSE)) %>% 
  mutate(c_trial_gender = scale(as.numeric(trial_gender),scale=FALSE)) 

final_prod <- final_prod %>%
  mutate(poli_party = factor(poli_party,ordered = FALSE)) %>%
  mutate(trial_gender = factor(gender,ordered = FALSE)) %>% 
  mutate(c_trial_gender = scale(as.numeric(trial_gender),scale=FALSE)) %>% 
  filter(incongruent == "real")

final_prod$poli_party = relevel(final_prod$poli_party, ref="Democrat")
final_prod$trial_gender = relevel(final_prod$trial_gender, ref="male")

prod_ultimate_model <- lmer(response_neutral~c_trial_gender*c_gender_total:subject_information.age + log_odds*poli_party*subject_information.age +log_odds:c_trial_gender + c_gender_total + (1 + c_trial_gender |workerid) + (1 + trial_gender|lexeme),data=final_prod)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning: Some predictor variables are on very different scales: consider rescaling

summary(prod_ultimate_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * c_gender_total:subject_information.age +  
    log_odds * poli_party * subject_information.age + log_odds:c_trial_gender +  
    c_gender_total + (1 + c_trial_gender | workerid) + (1 + trial_gender |      lexeme)
   Data: final_prod

REML criterion at convergence: 4824.7

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.79727 -0.65673 -0.03087  0.66013  2.93534 

Random effects:
 Groups   Name               Variance Std.Dev. Corr 
 workerid (Intercept)        0.01893  0.1376        
          c_trial_gender     0.01102  0.1050   -0.60
 lexeme   (Intercept)        0.08069  0.2841        
          trial_genderfemale 0.17420  0.4174   -0.86
 Residual                    0.12617  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                          Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                              5.759e-01  7.081e-02  1.812e+02   8.134  6.4e-14 ***
c_trial_gender                                           2.221e-01  9.748e-02  1.976e+01   2.278 0.033974 *  
log_odds                                                 5.477e-02  1.387e-02  1.797e+01   3.950 0.000941 ***
poli_partyNon-Partisan                                  -1.089e-01  9.487e-02  2.664e+02  -1.148 0.251993    
poli_partyRepublican                                    -2.044e-01  9.045e-02  2.689e+02  -2.259 0.024654 *  
subject_information.age                                  8.819e-04  2.285e-03  2.726e+02   0.386 0.699880    
c_gender_total                                           3.440e-03  2.545e-03  2.661e+02   1.352 0.177598    
c_gender_total:subject_information.age                  -2.005e-04  8.961e-05  2.707e+02  -2.237 0.026069 *  
log_odds:poli_partyNon-Partisan                          3.367e-07  6.193e-06  4.870e+03   0.054 0.956638    
log_odds:poli_partyRepublican                            3.053e-07  4.698e-06  4.860e+03   0.065 0.948180    
subject_information.age:log_odds                        -6.286e-08  1.349e-07  4.858e+03  -0.466 0.641315    
subject_information.age:poli_partyNon-Partisan           2.300e-03  3.343e-03  2.676e+02   0.688 0.492017    
subject_information.age:poli_partyRepublican             3.503e-03  3.358e-03  2.703e+02   1.043 0.297795    
c_trial_gender:log_odds                                  1.095e-01  2.773e-02  1.797e+01   3.949 0.000943 ***
c_trial_gender:c_gender_total:subject_information.age   -2.792e-05  2.762e-05  2.786e+02  -1.011 0.312885    
subject_information.age:log_odds:poli_partyNon-Partisan -3.890e-08  2.288e-07  4.874e+03  -0.170 0.865023    
subject_information.age:log_odds:poli_partyRepublican    5.592e-08  1.828e-07  4.858e+03   0.306 0.759711    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 17 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

fit warnings:
Some predictor variables are on very different scales: consider rescaling

final_prod <- final_prod %>% 
  mutate(c_age = scale(subject_information.age, scale=FALSE)) %>% 
  mutate(c_log_odds = scale(log_odds, scale=FALSE))

3 Visualisations

final_prod %>%
  group_by(log_odds,trial_gender,lexeme) %>%
  unique() %>%
  ggplot(aes(x=log_odds,fill=trial_gender))+
  geom_histogram()

`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

final_prod %>% 
  group_by(trial_gender,subject_information.age) %>% 
  summarise(mean_prod = mean(response_neutral)) %>% 
  ggplot(aes(x=subject_information.age,y=mean_prod)) + 
  geom_point() + 
  geom_smooth(method="lm")

`summarise()` has grouped output by 'trial_gender'. You can override using the `.groups` argument.

`geom_smooth()` using formula 'y ~ x'

final_prod %>% 
  ggplot(aes(x=subject_information.age,y=gender_total,color=poli_party)) + 
  geom_point() + 
  geom_smooth(method="lm")

`geom_smooth()` using formula 'y ~ x'

final_prod %>% 
  group_by(trial_gender,subject_information.party_alignment) %>% 
  summarise(mean_prod = mean(response_neutral)) %>% 
  ggplot(aes(x=subject_information.party_alignment,y=mean_prod,fill=as.factor(subject_information.party_alignment))) + 
  geom_bar(stat="identity") + 
  scale_fill_manual(values = bran_palette) +
    theme(legend.position = "none")

`summarise()` has grouped output by 'trial_gender'. You can override using the `.groups` argument.

final_prod %>% 
  group_by(workerid,poli_party) %>% 
  summarize(mean_prod = mean(response_neutral)) %>% 
  ggplot(aes(x=poli_party,y=mean_prod,fill=poli_party)) + 
  geom_dotplot(binaxis = "y",stackdir = "center",dotsize = 0.5) + 
  stat_summary(fun.y=mean, geom="point", shape=18,
                 size=5, color="red") + 
  theme(legend.position = "none") + 
  scale_fill_manual(values=bran_palette) + 
  scale_x_discrete(limits=c("Democrat","Non-Partisan","Republican")) + 
  labs(x="Participant Political Party",y="Proportion of Neutral Responses")

`summarise()` has grouped output by 'workerid'. You can override using the `.groups` argument.

Warning: `fun.y` is deprecated. Use `fun` instead.
Bin width defaults to 1/30 of the range of the data. Pick better value with `binwidth`.

ggsave("prod-dot-party.png", width=7,height=5,path='/Users/branpap/Desktop/gender_ideology/talks_and_papers/qp_paper/figures')

Bin width defaults to 1/30 of the range of the data. Pick better value with `binwidth`.

final_prod %>% 
  group_by(trial_gender,gender_total,young_old) %>% 
  summarise(mean_prod = mean(response_neutral)) %>% 
  ggplot(aes(x=gender_total,y=mean_prod,color=young_old)) + 
  geom_point() + 
  geom_smooth(method="lm")

`summarise()` has grouped output by 'trial_gender', 'gender_total'. You can override using the `.groups` argument.

`geom_smooth()` using formula 'y ~ x'

3.1 Production Proportion Results

prod_all %>% mutate(response_gender = case_when( response_gender == “male/neutral” ~ “neutral”, TRUE ~ response_gender )) %>% ggplot() + geom_mosaic(aes(x=product(subject_information.party_alignment), fill=response_gender)) + scale_fill_manual(values=bran_palette) + labs(x=“Political Alignment”,y=“Response Gender Proportions”, fill=“Response Gender”, title=“Proportion of Response Genders by Gender Seen (facet) and political alignment”) + facet_wrap(~gender)

test <- prod_all %>% 
  mutate(response_gender = case_when(
    response_gender == "male/neutral" ~ "neutral",
    TRUE ~ response_gender
  )) %>%
  select(workerid,gender,response_gender,subject_information.party_alignment) %>% 
  rename(trial_gender = gender) %>% 
  mutate(trial_gender = as.factor(trial_gender),
         response_gender = as.factor(response_gender))

test_d <- test %>% 
  mutate(id = as.factor(row_number())) %>% 
  expand_grid(c("male","female","neutral"),test$id) %>% 
  rename(response_old = response_gender)

colnames(test_d)[6] <- "response_gender"

test_d <- test_d %>% 
  group_by(id) %>% 
  mutate(response = case_when(response_gender == response_old ~ 1,TRUE ~ 0)) %>% 
  select(-response_old)

dodge = position_dodge(0.9)

blah <- test_d %>% 
  group_by(response_gender,trial_gender,subject_information.party_alignment) %>% 
  summarise(participants = n_distinct(workerid))

`summarise()` has grouped output by 'response_gender', 'trial_gender'. You can override using the `.groups` argument.

test_d %>% 
  group_by(response_gender,trial_gender,subject_information.party_alignment) %>% 
  summarize(proportion = mean(response),
            CI.Low = ci.low(response),
            CI.High = ci.high(response),
            obs = n(),
            participants = n_distinct(workerid)) %>%
  rename(party_alignment = subject_information.party_alignment) %>% 
  ungroup() %>% 
  mutate(YMin = proportion - CI.Low,
         YMax = proportion + CI.High) %>% 
  ggplot(aes(x=party_alignment, 
             y=proportion, 
             fill=response_gender,
             alpha = participants)) + 
  geom_bar(stat="identity",
           position = dodge) + 
  geom_errorbar(aes(ymin = YMin, ymax=YMax),
                position = dodge,
                width=0.25) + 
  facet_wrap(~trial_gender) + 
  scale_fill_manual(values = bran_palette) + 
  scale_alpha(range=c(0.4,1)) + 
  labs(y="Proportion of Responses",x="<-- Republican           Participant Party Alignment           Democrat -->", fill="Response Form Gender", alpha="# of Participants")

`summarise()` has grouped output by 'response_gender', 'trial_gender'. You can override using the `.groups` argument.

3.2 Same but by ideology

test_2 <- prod_all %>% 
  mutate(response_gender = case_when(
    response_gender == "male/neutral" ~ "neutral",
    TRUE ~ response_gender
  )) %>%
  select(workerid,gender,response_gender,gender_total,poli_party) %>% 
  mutate(ideology = cut_number(gender_total,n=3,labels=c("progressive","moderate","conservative"))) %>% 
  rename(trial_gender = gender) %>% 
  mutate(trial_gender = as.factor(trial_gender),
         response_gender = as.factor(response_gender))

test_e <- test_2 %>% 
  mutate(id = as.factor(row_number())) %>% 
  expand_grid(c("male","female","neutral"),test$id) %>% 
  rename(response_old = response_gender)

colnames(test_e)[8] <- "response_gender"

test_e <- test_e %>% 
  group_by(id) %>% 
  mutate(response = case_when(response_gender == response_old ~ 1,TRUE ~ 0)) %>% 
  select(-response_old)

blah2<- test_e %>% 
  group_by(response_gender,trial_gender,ideology,poli_party) %>% 
  summarise(participants = n_distinct(workerid))

`summarise()` has grouped output by 'response_gender', 'trial_gender', 'ideology'. You can override using the `.groups` argument.

test_e %>% 
  group_by(response_gender,trial_gender,ideology,poli_party) %>% 
  summarize(proportion = mean(response),
            CI.Low = ci.low(response),
            CI.High = ci.high(response),
            obs = n(),
            participants = n_distinct(workerid)) %>%
  ungroup() %>% 
  mutate(YMin = proportion - CI.Low,
         YMax = proportion + CI.High) %>% 
  ggplot(aes(x=ideology, 
             y=proportion, 
             fill=response_gender,
             alpha=as.numeric(participants))) + 
  geom_bar(stat="identity",
           position = dodge) + 
  geom_errorbar(aes(ymin = YMin, ymax=YMax),
                position = dodge,
                width=0.25) + 
  facet_grid(poli_party~trial_gender) + 
  scale_fill_manual(values = sec_palette, limits=c("male","female","neutral")) + 
  scale_alpha(range=c(0.4,1)) + 
  labs(y="Proportion of Responses",x="Gender Ideology", fill="Response Form Gender",alpha = "# of Participants") + 
  theme(text=element_text(size=14)) + 
  theme(axis.text.x = element_text(angle=20, vjust=.8)) + 
  theme(legend.position = "top", legend.text = element_text(size=10), legend.title = element_text(size=10))

`summarise()` has grouped output by 'response_gender', 'trial_gender', 'ideology'. You can override using the `.groups` argument.

# + 
#  scale_x_discrete(sec.axis = sec_axis(~ . , name = "Gender Seen", breaks = NULL))

ggsave("prod-3x2x3.png", width=7,height=5,path='/Users/branpap/Desktop/gender_ideology/talks_and_papers/qp_paper/figures')

4 Incongruent Response Analysis (Auxiliary)

incongruent <- prod_all %>%
  filter(incongruent == "incongruent_ftm" | incongruent == "incongruent_mtf")

incongruent%>%
  group_by(workerid,incongruent) %>%
  count() %>%
  arrange(n) %>%
  ggplot(aes(x=n,fill=incongruent)) + 
  geom_histogram()

`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

4.1 New Tests

simple_ideology <- lmer(response_neutral~c_trial_gender*c_gender_total + (1|workerid) + (1|lexeme),data=final_prod)

summary(simple_ideology)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * c_gender_total + (1 | workerid) +      (1 | lexeme)
   Data: final_prod

REML criterion at convergence: 6058

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.4697 -0.8078  0.0840  0.7969  2.4395 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.02106  0.1451  
 lexeme   (Intercept) 0.05248  0.2291  
 Residual             0.17310  0.4161  
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                    5.202e-01  5.230e-02  2.012e+01   9.947 3.26e-09 ***
c_trial_gender                 1.030e-01  1.161e-02  4.905e+03   8.865  < 2e-16 ***
c_gender_total                -3.962e-03  6.698e-04  2.717e+02  -5.915 9.94e-09 ***
c_trial_gender:c_gender_total -4.559e-04  7.402e-04  4.906e+03  -0.616    0.538    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ c_gnd_
c_tril_gndr -0.004              
c_gendr_ttl  0.001 -0.003       
c_trl_gn:__ -0.001  0.014 -0.022

simple_political <- lmer(response_neutral~c_trial_gender*poli_party + (1|workerid) + (1|lexeme),data=final_prod)

summary(simple_political)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party + (1 | workerid) +      (1 | lexeme)
   Data: final_prod

REML criterion at convergence: 6049.4

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.47590 -0.80379  0.08785  0.79611  2.41957 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.02062  0.1436  
 lexeme   (Intercept) 0.05256  0.2293  
 Residual             0.17315  0.4161  
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                        Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                              0.58534    0.05349   21.93086  10.944 2.37e-10 ***
c_trial_gender                           0.09270    0.01692 4905.63445   5.478 4.51e-08 ***
poli_partyNon-Partisan                  -0.06160    0.03282  266.81294  -1.877   0.0617 .  
poli_partyRepublican                    -0.14271    0.02254  269.18107  -6.332 1.01e-09 ***
c_trial_gender:poli_partyNon-Partisan    0.01450    0.03623 4895.73962   0.400   0.6891    
c_trial_gender:poli_partyRepublican      0.02061    0.02501 4903.49828   0.824   0.4098    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ pl_N-P pl_prR c__:_N
c_tril_gndr -0.006                            
pl_prtyNn-P -0.133  0.010                     
pl_prtyRpbl -0.193  0.015  0.314              
c_trl_:_N-P  0.003 -0.467 -0.015 -0.007       
c_trl_gn:_R  0.004 -0.676 -0.007 -0.020  0.316

simple_social <- lmer(response_neutral~c_trial_gender*poli_party + c_trial_gender:c_gender_total + c_gender_total + (1|workerid) + (1|lexeme),data=final_prod)

summary(simple_social)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party + c_trial_gender:c_gender_total +  
    c_gender_total + (1 | workerid) + (1 | lexeme)
   Data: final_prod

REML criterion at convergence: 6060.6

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.46891 -0.79930  0.08565  0.80170  2.44193 

Random effects:
 Groups   Name        Variance Std.Dev.
 workerid (Intercept) 0.01941  0.1393  
 lexeme   (Intercept) 0.05250  0.2291  
 Residual             0.17313  0.4161  
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                        Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                            5.679e-01  5.360e-02  2.217e+01  10.595 3.83e-10 ***
c_trial_gender                         8.624e-02  1.796e-02  4.914e+03   4.802 1.62e-06 ***
poli_partyNon-Partisan                -4.874e-02  3.236e-02  2.665e+02  -1.506 0.133129    
poli_partyRepublican                  -1.035e-01  2.476e-02  2.724e+02  -4.180 3.93e-05 ***
c_gender_total                        -2.578e-03  7.312e-04  2.732e+02  -3.525 0.000496 ***
c_trial_gender:poli_partyNon-Partisan  1.926e-02  3.651e-02  4.897e+03   0.528 0.597781    
c_trial_gender:poli_partyRepublican    3.512e-02  2.826e-02  4.914e+03   1.243 0.214051    
c_trial_gender:c_gender_total         -9.583e-04  8.374e-04  4.914e+03  -1.144 0.252549    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ pl_N-P pl_prR c_gnd_ c__:_N c__:_R
c_tril_gndr -0.008                                          
pl_prtyNn-P -0.139  0.013                                   
pl_prtyRpbl -0.210  0.023  0.330                            
c_gendr_ttl  0.093 -0.018 -0.114 -0.453                     
c_trl_:_N-P  0.004 -0.478 -0.016 -0.010  0.007              
c_trl_gn:_R  0.007 -0.720 -0.010 -0.029  0.021  0.335       
c_trl_gn:__ -0.005  0.335  0.007  0.021 -0.031 -0.123 -0.466

simple_social_2 <- lmer(response_neutral~c_trial_gender + poli_party + c_gender_total + (1 + c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

summary(simple_social_2)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender + poli_party + c_gender_total +  
    (1 + c_trial_gender | workerid) + (1 + c_trial_gender | lexeme)
   Data: final_prod

REML criterion at convergence: 4621.1

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.79732 -0.65815 -0.03438  0.66816  2.94329 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01908  0.1381        
          c_trial_gender 0.01097  0.1047   -0.57
 lexeme   (Intercept)    0.04990  0.2234        
          c_trial_gender 0.18633  0.4317   0.27 
 Residual                0.12617  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                         Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)             5.712e-01  5.204e-02  2.194e+01  10.976 2.23e-10 ***
c_trial_gender          9.018e-02  9.724e-02  1.916e+01   0.927 0.365243    
poli_partyNon-Partisan -4.026e-02  2.953e-02  2.662e+02  -1.363 0.174020    
poli_partyRepublican   -1.013e-01  2.254e-02  2.694e+02  -4.497 1.03e-05 ***
c_gender_total         -2.215e-03  6.657e-04  2.705e+02  -3.328 0.000997 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ pl_N-P pl_prR
c_tril_gndr  0.248                     
pl_prtyNn-P -0.130  0.000              
pl_prtyRpbl -0.197  0.000  0.329       
c_gendr_ttl  0.087  0.000 -0.113 -0.450

mixed_social <- lmer(response_neutral~c_trial_gender + poli_party*c_gender_total + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

summary(mixed_social)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender + poli_party * c_gender_total +  
    (1 + c_trial_gender | workerid) + (1 + c_trial_gender | lexeme)
   Data: final_prod

REML criterion at convergence: 4627

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.79113 -0.65688 -0.03632  0.66294  2.93008 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01779  0.1334        
          c_trial_gender 0.01096  0.1047   -0.58
 lexeme   (Intercept)    0.04996  0.2235        
          c_trial_gender 0.18640  0.4317   0.27 
 Residual                0.12616  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                        Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                            5.508e-01  5.222e-02  2.218e+01  10.549 4.15e-10 ***
c_trial_gender                         9.028e-02  9.726e-02  1.916e+01   0.928 0.364847    
poli_partyNon-Partisan                -1.604e-02  2.960e-02  2.654e+02  -0.542 0.588412    
poli_partyRepublican                  -1.003e-01  2.221e-02  2.678e+02  -4.516 9.45e-06 ***
c_gender_total                        -5.234e-03  9.913e-04  2.730e+02  -5.280 2.64e-07 ***
poli_partyNon-Partisan:c_gender_total  5.260e-03  2.302e-03  2.645e+02   2.285 0.023097 *  
poli_partyRepublican:c_gender_total    5.295e-03  1.367e-03  2.690e+02   3.874 0.000134 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ pl_N-P pl_prR c_gnd_ p_N-P:
c_tril_gndr  0.249                                   
pl_prtyNn-P -0.143  0.000                            
pl_prtyRpbl -0.191  0.000  0.336                     
c_gendr_ttl  0.129  0.000 -0.227 -0.303              
pl_prN-P:__ -0.055  0.000  0.209  0.130 -0.430       
pl_prtyR:__ -0.093  0.000  0.164 -0.028 -0.725  0.312

interacting_social <- lmer(response_neutral~c_trial_gender*poli_party*c_gender_total + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

summary(interacting_social)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party * c_gender_total +  
    (1 + c_trial_gender | workerid) + (1 + c_trial_gender | lexeme)
   Data: final_prod

REML criterion at convergence: 4667.9

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.80336 -0.65398 -0.03729  0.66282  2.92639 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01782  0.1335        
          c_trial_gender 0.01134  0.1065   -0.57
 lexeme   (Intercept)    0.04991  0.2234        
          c_trial_gender 0.18649  0.4318   0.27 
 Residual                0.12616  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                       Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                           5.543e-01  5.232e-02  2.239e+01  10.593 3.45e-10 ***
c_trial_gender                                        7.555e-02  9.854e-02  2.015e+01   0.767 0.452143    
poli_partyNon-Partisan                               -2.095e-02  3.087e-02  2.669e+02  -0.679 0.497836    
poli_partyRepublican                                 -1.086e-01  2.320e-02  2.711e+02  -4.681 4.52e-06 ***
c_gender_total                                       -5.085e-03  1.037e-03  2.776e+02  -4.903 1.61e-06 ***
c_trial_gender:poli_partyNon-Partisan                 2.133e-02  3.847e-02  2.623e+02   0.554 0.579766    
c_trial_gender:poli_partyRepublican                   3.589e-02  2.910e-02  2.720e+02   1.233 0.218588    
c_trial_gender:c_gender_total                        -6.368e-04  1.313e-03  2.851e+02  -0.485 0.628037    
poli_partyNon-Partisan:c_gender_total                 5.423e-03  2.399e-03  2.656e+02   2.260 0.024620 *  
poli_partyRepublican:c_gender_total                   5.474e-03  1.429e-03  2.728e+02   3.832 0.000158 ***
c_trial_gender:poli_partyNon-Partisan:c_gender_total -7.569e-04  2.985e-03  2.593e+02  -0.254 0.799991    
c_trial_gender:poli_partyRepublican:c_gender_total   -7.874e-04  1.796e-03  2.756e+02  -0.438 0.661472    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ pl_N-P pl_prR c_gnd_ c_t_:_N-P c_t_:_R c__:__ p_N-P: p_R:__ c__:_N-P:
c_tril_gndr  0.233                                                                             
pl_prtyNn-P -0.150  0.029                                                                      
pl_prtyRpbl -0.199  0.039  0.338                                                               
c_gendr_ttl  0.135 -0.028 -0.229 -0.305                                                        
c_trl_:_N-P  0.044 -0.101 -0.283 -0.100  0.072                                                 
c_trl_gn:_R  0.058 -0.134 -0.099 -0.289  0.095  0.343                                          
c_trl_gn:__ -0.042  0.094  0.071  0.094 -0.294 -0.241    -0.319                                
pl_prN-P:__ -0.058  0.012  0.211  0.132 -0.432 -0.063    -0.041   0.127                        
pl_prtyR:__ -0.098  0.020  0.166 -0.026 -0.726 -0.052     0.003   0.213  0.314                 
c__:_N-P:__  0.018 -0.041 -0.063 -0.041  0.129  0.218     0.140  -0.438 -0.282 -0.094          
c_tr_:_R:__  0.031 -0.069 -0.052  0.003  0.214  0.175    -0.014  -0.730 -0.093 -0.291  0.321   

reps_only <- final_prod %>% 
  filter(poli_party == "Republican")

dems_only <- final_prod %>% 
  filter(poli_party == "Democrat")

ind_only <- final_prod %>% 
  filter(poli_party == "Non-Partisan")

rep_model <- lmer(response_neutral~c_trial_gender*c_gender_total + c_log_odds:c_age + c_log_odds + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=reps_only)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning: Some predictor variables are on very different scales: consider rescaling

summary(rep_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * c_gender_total + c_log_odds:c_age +  
    c_log_odds + (1 + c_trial_gender | workerid) + (1 + c_trial_gender |      lexeme)
   Data: reps_only

REML criterion at convergence: 1859.5

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.77436 -0.55182 -0.08587  0.50309  2.83991 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.010895 0.10438       
          c_trial_gender 0.009943 0.09972  -0.84
 lexeme   (Intercept)    0.050303 0.22428       
          c_trial_gender 0.199463 0.44661  0.55 
 Residual                0.121913 0.34916       
Number of obs: 2062, groups:  workerid, 108; lexeme, 20

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                    4.423e-01  5.222e-02  2.104e+01   8.470 3.21e-08 ***
c_trial_gender                 1.700e-01  1.034e-01  2.033e+01   1.644  0.11562    
c_gender_total                 4.768e-04  8.286e-04  1.062e+02   0.575  0.56627    
c_log_odds                     1.209e-05  3.565e-06  1.804e+01   3.392  0.00324 ** 
c_trial_gender:c_gender_total -1.350e-03  1.194e-03  1.073e+02  -1.131  0.26052    
c_log_odds:c_age              -9.959e-09  1.211e-07  1.930e+03  -0.082  0.93449    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ c_gnd_ c_lg_d c__:__
c_tril_gndr  0.485                            
c_gendr_ttl -0.135  0.037                     
c_log_odds   0.000  0.163  0.002              
c_trl_gn:__  0.051 -0.099 -0.374 -0.003       
c_lg_dds:c_ -0.002  0.002  0.007  0.018 -0.029
fit warnings:
Some predictor variables are on very different scales: consider rescaling

ind_model <- lmer(response_neutral~c_trial_gender*c_gender_total + c_log_odds:c_age + c_log_odds + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=ind_only)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning: Some predictor variables are on very different scales: consider rescaling

summary(ind_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * c_gender_total + c_log_odds:c_age +  
    c_log_odds + (1 + c_trial_gender | workerid) + (1 + c_trial_gender |      lexeme)
   Data: ind_only

REML criterion at convergence: 748.3

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.41626 -0.66027 -0.02928  0.60077  2.63360 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.020378 0.14275       
          c_trial_gender 0.003495 0.05912  -0.92
 lexeme   (Intercept)    0.051664 0.22730       
          c_trial_gender 0.171858 0.41456  0.29 
 Residual                0.126586 0.35579       
Number of obs: 679, groups:  workerid, 35; lexeme, 20

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                    5.344e-01  5.810e-02  2.628e+01   9.198 1.06e-09 ***
c_trial_gender                 1.399e-01  9.963e-02  1.974e+01   1.404   0.1759    
c_gender_total                -1.402e-05  2.286e-03  3.351e+01  -0.006   0.9951    
c_log_odds                     9.207e-06  4.455e-06  1.809e+01   2.067   0.0534 .  
c_trial_gender:c_gender_total -1.401e-03  2.421e-03  3.118e+01  -0.578   0.5671    
c_log_odds:c_age              -1.174e-07  1.856e-07  6.211e+02  -0.632   0.5273    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ c_gnd_ c_lg_d c__:__
c_tril_gndr  0.193                            
c_gendr_ttl  0.071 -0.009                     
c_log_odds   0.000  0.210  0.013              
c_trl_gn:__ -0.020  0.041 -0.280 -0.023       
c_lg_dds:c_  0.011  0.001  0.026 -0.054  0.017
fit warnings:
Some predictor variables are on very different scales: consider rescaling

dem_model <- lmer(response_neutral~c_trial_gender*c_gender_total + c_log_odds:c_age + c_log_odds + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=dems_only)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning: Some predictor variables are on very different scales: consider rescaling

summary(dem_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * c_gender_total + c_log_odds:c_age +  
    c_log_odds + (1 + c_trial_gender | workerid) + (1 + c_trial_gender |      lexeme)
   Data: dems_only

REML criterion at convergence: 2347.9

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.75166 -0.67859  0.04751  0.68489  2.74682 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.02295  0.1515        
          c_trial_gender 0.01539  0.1240   -0.36
 lexeme   (Intercept)    0.03322  0.1823        
          c_trial_gender 0.16871  0.4107   0.30 
 Residual                0.12702  0.3564        
Number of obs: 2438, groups:  workerid, 128; lexeme, 20

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                    5.563e-01  4.420e-02  2.416e+01  12.587 4.19e-12 ***
c_trial_gender                 1.181e-01  9.582e-02  2.058e+01   1.233   0.2316    
c_gender_total                -5.045e-03  1.140e-03  1.313e+02  -4.424 2.01e-05 ***
c_log_odds                     1.014e-05  3.782e-06  1.794e+01   2.682   0.0152 *  
c_trial_gender:c_gender_total -8.479e-04  1.386e-03  1.348e+02  -0.612   0.5417    
c_log_odds:c_age              -7.587e-08  1.359e-07  2.300e+03  -0.558   0.5766    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) c_trl_ c_gnd_ c_lg_d c__:__
c_tril_gndr  0.251                            
c_gendr_ttl  0.176 -0.023                     
c_log_odds   0.001  0.186  0.001              
c_trl_gn:__ -0.041  0.102 -0.215  0.000       
c_lg_dds:c_  0.006  0.005  0.014  0.015  0.056
fit warnings:
Some predictor variables are on very different scales: consider rescaling

log_odds_model <- lmer(response_neutral~c_trial_gender*poli_party*c_gender_total + log_odds + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge with max|grad| = 0.00335037 (tol = 0.002, component 1)
Warning: Some predictor variables are on very different scales: consider rescaling

summary(log_odds_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party * c_gender_total +  
    log_odds + (1 + c_trial_gender | workerid) + (1 + c_trial_gender |      lexeme)
   Data: final_prod

REML criterion at convergence: 4683.7

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.79626 -0.65492 -0.03814  0.66495  2.92374 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01782  0.1335        
          c_trial_gender 0.01135  0.1066   -0.57
 lexeme   (Intercept)    0.04122  0.2030        
          c_trial_gender 0.18087  0.4253   0.42 
 Residual                0.12615  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                       Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                           5.288e-01  4.877e-02  2.377e+01  10.842 1.11e-10 ***
c_trial_gender                                        1.265e-01  9.864e-02  2.081e+01   1.283 0.213686    
poli_partyNon-Partisan                               -2.097e-02  3.087e-02  2.669e+02  -0.679 0.497598    
poli_partyRepublican                                 -1.086e-01  2.320e-02  2.711e+02  -4.681 4.52e-06 ***
c_gender_total                                       -5.084e-03  1.037e-03  2.776e+02  -4.902 1.62e-06 ***
log_odds                                              1.085e-05  3.690e-06  1.799e+01   2.939 0.008773 ** 
c_trial_gender:poli_partyNon-Partisan                 2.137e-02  3.847e-02  2.623e+02   0.555 0.579130    
c_trial_gender:poli_partyRepublican                   3.589e-02  2.911e-02  2.720e+02   1.233 0.218650    
c_trial_gender:c_gender_total                        -6.374e-04  1.313e-03  2.851e+02  -0.485 0.627783    
poli_partyNon-Partisan:c_gender_total                 5.431e-03  2.399e-03  2.656e+02   2.264 0.024408 *  
poli_partyRepublican:c_gender_total                   5.476e-03  1.429e-03  2.728e+02   3.833 0.000157 ***
c_trial_gender:poli_partyNon-Partisan:c_gender_total -7.718e-04  2.985e-03  2.594e+02  -0.259 0.796186    
c_trial_gender:poli_partyRepublican:c_gender_total   -7.932e-04  1.797e-03  2.756e+02  -0.442 0.659166    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 13 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

fit warnings:
Some predictor variables are on very different scales: consider rescaling
optimizer (nloptwrap) convergence code: 0 (OK)
Model failed to converge with max|grad| = 0.00335037 (tol = 0.002, component 1)

neutral_logel <- lmer(response_neutral~c_trial_gender*poli_party*c_gender_total + log_all_neutral + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge with max|grad| = 0.00498731 (tol = 0.002, component 1)

summary(neutral_logel)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party * c_gender_total +  
    log_all_neutral + (1 + c_trial_gender | workerid) + (1 +      c_trial_gender | lexeme)
   Data: final_prod

REML criterion at convergence: 4669

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.80991 -0.65652 -0.03678  0.66293  2.93042 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01782  0.1335        
          c_trial_gender 0.01132  0.1064   -0.57
 lexeme   (Intercept)    0.03798  0.1949        
          c_trial_gender 0.18634  0.4317   -0.13
 Residual                0.12616  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                       Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                           5.159e-01  4.798e-02  2.166e+01  10.753 3.79e-10 ***
c_trial_gender                                        7.568e-02  9.850e-02  2.016e+01   0.768 0.451181    
poli_partyNon-Partisan                               -2.093e-02  3.087e-02  2.669e+02  -0.678 0.498364    
poli_partyRepublican                                 -1.086e-01  2.320e-02  2.711e+02  -4.681 4.52e-06 ***
c_gender_total                                       -5.086e-03  1.037e-03  2.776e+02  -4.904 1.60e-06 ***
log_all_neutral                                       5.471e-02  1.811e-02  1.801e+01   3.020 0.007347 ** 
c_trial_gender:poli_partyNon-Partisan                 2.126e-02  3.846e-02  2.623e+02   0.553 0.580861    
c_trial_gender:poli_partyRepublican                   3.591e-02  2.910e-02  2.720e+02   1.234 0.218244    
c_trial_gender:c_gender_total                        -6.295e-04  1.313e-03  2.851e+02  -0.480 0.631943    
poli_partyNon-Partisan:c_gender_total                 5.428e-03  2.399e-03  2.656e+02   2.262 0.024499 *  
poli_partyRepublican:c_gender_total                   5.473e-03  1.429e-03  2.728e+02   3.831 0.000158 ***
c_trial_gender:poli_partyNon-Partisan:c_gender_total -7.653e-04  2.984e-03  2.593e+02  -0.256 0.797780    
c_trial_gender:poli_partyRepublican:c_gender_total   -7.964e-04  1.796e-03  2.756e+02  -0.443 0.657753    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 13 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

optimizer (nloptwrap) convergence code: 0 (OK)
Model failed to converge with max|grad| = 0.00498731 (tol = 0.002, component 1)

norm_model <- lmer(response_neutral~c_trial_gender*poli_party*c_gender_total + form_norm + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

boundary (singular) fit: see ?isSingular

summary(norm_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party * c_gender_total +  
    form_norm + (1 + c_trial_gender | workerid) + (1 + c_trial_gender |      lexeme)
   Data: final_prod

REML criterion at convergence: 3479.1

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.9483 -0.5514 -0.0203  0.5277  3.9314 

Random effects:
 Groups   Name           Variance Std.Dev. Corr
 workerid (Intercept)    0.01208  0.1099       
          c_trial_gender 0.01820  0.1349   1.00
 lexeme   (Intercept)    0.05079  0.2254       
          c_trial_gender 0.18936  0.4352   0.85
 Residual                0.10102  0.3178       
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                       Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                           1.259e+00  5.478e-02  2.609e+01  22.978  < 2e-16 ***
c_trial_gender                                       -3.239e-01  9.978e-02  2.064e+01  -3.246 0.003930 ** 
poli_partyNon-Partisan                               -1.295e-02  2.601e-02  2.733e+02  -0.498 0.619052    
poli_partyRepublican                                 -8.466e-02  1.956e-02  2.780e+02  -4.329 2.10e-05 ***
c_gender_total                                       -4.418e-03  8.735e-04  2.841e+02  -5.058 7.62e-07 ***
form_norm                                            -1.742e-01  4.197e-03  4.622e+03 -41.497  < 2e-16 ***
c_trial_gender:poli_partyNon-Partisan                -2.021e-03  3.918e-02  3.141e+02  -0.052 0.958890    
c_trial_gender:poli_partyRepublican                  -6.197e-02  2.963e-02  3.270e+02  -2.092 0.037235 *  
c_trial_gender:c_gender_total                        -4.796e-03  1.329e-03  3.398e+02  -3.609 0.000353 ***
poli_partyNon-Partisan:c_gender_total                 4.455e-03  2.022e-03  2.720e+02   2.203 0.028425 *  
poli_partyRepublican:c_gender_total                   4.352e-03  1.204e-03  2.796e+02   3.615 0.000357 ***
c_trial_gender:poli_partyNon-Partisan:c_gender_total  4.288e-03  3.045e-03  3.123e+02   1.408 0.160020    
c_trial_gender:poli_partyRepublican:c_gender_total    4.116e-03  1.824e-03  3.292e+02   2.256 0.024705 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 13 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular

age_model <- lmer(response_neutral~c_trial_gender*poli_party*c_gender_total + c_log_odds:c_age + c_log_odds + (1+ c_trial_gender|workerid) + (1 + c_trial_gender|lexeme),data=final_prod)

Warning: Some predictor variables are on very different scales: consider rescaling
Warning: Some predictor variables are on very different scales: consider rescaling

summary(age_model)

Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest']
Formula: response_neutral ~ c_trial_gender * poli_party * c_gender_total +  
    c_log_odds:c_age + c_log_odds + (1 + c_trial_gender | workerid) +      (1 + c_trial_gender | lexeme)
   Data: final_prod

REML criterion at convergence: 4714

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-2.7976 -0.6542 -0.0381  0.6670  2.9243 

Random effects:
 Groups   Name           Variance Std.Dev. Corr 
 workerid (Intercept)    0.01784  0.1336        
          c_trial_gender 0.01133  0.1064   -0.58
 lexeme   (Intercept)    0.04124  0.2031        
          c_trial_gender 0.18074  0.4251   0.42 
 Residual                0.12617  0.3552        
Number of obs: 5179, groups:  workerid, 271; lexeme, 20

Fixed effects:
                                                       Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                                           5.544e-01  4.800e-02  2.283e+01  11.549 5.19e-11 ***
c_trial_gender                                        1.264e-01  9.861e-02  2.083e+01   1.282 0.214078    
poli_partyNon-Partisan                               -2.105e-02  3.088e-02  2.668e+02  -0.682 0.496079    
poli_partyRepublican                                 -1.084e-01  2.321e-02  2.711e+02  -4.672 4.71e-06 ***
c_gender_total                                       -5.091e-03  1.038e-03  2.776e+02  -4.906 1.58e-06 ***
c_log_odds                                            1.083e-05  3.690e-06  1.800e+01   2.935 0.008845 ** 
c_trial_gender:poli_partyNon-Partisan                 2.104e-02  3.847e-02  2.624e+02   0.547 0.584884    
c_trial_gender:poli_partyRepublican                   3.602e-02  2.910e-02  2.720e+02   1.238 0.216897    
c_trial_gender:c_gender_total                        -6.730e-04  1.314e-03  2.857e+02  -0.512 0.608841    
poli_partyNon-Partisan:c_gender_total                 5.411e-03  2.401e-03  2.656e+02   2.254 0.025002 *  
poli_partyRepublican:c_gender_total                   5.481e-03  1.429e-03  2.728e+02   3.835 0.000156 ***
c_log_odds:c_age                                     -6.213e-08  8.104e-08  4.932e+03  -0.767 0.443313    
c_trial_gender:poli_partyNon-Partisan:c_gender_total -7.358e-04  2.985e-03  2.594e+02  -0.247 0.805458    
c_trial_gender:poli_partyRepublican:c_gender_total   -7.407e-04  1.797e-03  2.764e+02  -0.412 0.680564    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 14 > 12.
Use print(x, correlation=TRUE)  or
    vcov(x)        if you need it

fit warnings:
Some predictor variables are on very different scales: consider rescaling

final_prod %>% 
  filter(log_odds >-40) %>% 
  filter(log_odds <35) %>% 
  group_by(poli_party,young_old,lexeme,log_odds,trial_gender) %>% 
  summarize(neutral_proportion = mean(response_neutral)) %>% 
  ggplot(aes(x=log_odds,y=neutral_proportion, color=young_old,label=lexeme)) + 
  geom_point() + 
  geom_text() +
  geom_smooth(method='lm') + 
  facet_grid(trial_gender~poli_party)

`summarise()` has grouped output by 'poli_party', 'young_old', 'lexeme', 'log_odds'. You can override using the `.groups` argument.

`geom_smooth()` using formula 'y ~ x'

final_prod %>% 
  filter(log_odds >-40) %>% 
  filter(log_odds <35) %>% 
  group_by(poli_party,young_old,lexeme,log_all_neutral) %>% 
  summarize(neutral_proportion = mean(response_neutral)) %>% 
  ggplot(aes(x=log_all_neutral,y=neutral_proportion, color=young_old)) + 
  geom_point() + 
  geom_smooth(method='lm') + 
  facet_wrap(~poli_party)

`summarise()` has grouped output by 'poli_party', 'young_old', 'lexeme'. You can override using the `.groups` argument.

`geom_smooth()` using formula 'y ~ x'

final_prod %>% 
filter(lexeme == "hero") %>% 
  select(gender,lexeme,form,response_neutral,log_odds)

hero <- final_prod %>% 
  filter(lexeme=="hero")

table(hero$poli_party,hero$response_neutral,hero$trial_gender,hero$young_old)

, ,  = male,  = old

              
                0  1
  Democrat      0  2
  Non-Partisan  0  2
  Republican    0  2

, ,  = female,  = old

              
                0  1
  Democrat      2  1
  Non-Partisan  1  0
  Republican    3  0

, ,  = male,  = young

              
                0  1
  Democrat      0 62
  Non-Partisan  0 19
  Republican    0 55

, ,  = female,  = young

              
                0  1
  Democrat     33 28
  Non-Partisan  9  4
  Republican   30 15

final_prod %>% 
  group_by(workerid,poli_party) %>% 
  unique()

4.1.1 Comments

final_prod %>% 
  filter(!is.na(subject_information.comments))

Error: Problem with `filter()` input `..1`.
ℹ Input `..1` is `!is.na(subject_information.comments)`.
x object 'subject_information.comments' not found
Run `rlang::last_error()` to see where the error occurred.

---

1. It is my hope and intention that this color palette be color-blind friendly. If you have accessibility concerns, please do not hesitate to reach out to me!↩︎

2. https://www.english-corpora.org/coca/↩︎

3. https://adfontesmedia.com↩︎